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Background / Context: 

Based on evidence derived from studies conducted mostly in the United States, many 
low- and middle-income countries are investing in early childhood education (ECE), with high 
expectations that it will improve academic outcomes, increase human capital, promote economic 
growth and reduce economic inequality. The impacts of ECE interventions depend on their 
quality (Camilli, Vargas, Ryan, & Barnett, 2010; Yoshikawa, 2014). An increasing amount of 
attention is being paid to quality in research and practice (Chavan, Yoshikawa & Bhadur, 2013; 
Sachs & Weiland, 2010), and to teacher professional development as a means to improve quality. 

In Chile, there has been a great expansion of ECE provision. Still, children’s language 
skills are below average, and there is a significant gap between socioeconomic groups. Several 
studies suggest that the quality of Chilean preschool environments may not be sufficient to 
address the disadvantages of low-income Chilean schoolchildren (Eyzaguirre & Le Foulon, 

2001; Manzi, Strasser, San Martin, &Contreras, 2008; Noboa-Hidalgo & Urzua, 2012). There is 
a need for and interest in professional development programs and methodologies that enable 
teachers to learn, develop and deliver high quality educational interactions and experiences. 

Un Buen Comienzo (A Good Start, UBC) was a cluster-randomized experiment of a two- 
year professional development program conducted in Chile that aimed to improve the quality of 
public preschool education and the outcomes of participating children. UBC provided 12 
monthly workshops and 24 biweekly in-classroom teacher coaching sessions to promote 
adoption of strategies to improve children’s language and literacy, with supporting interventions 
for children’s health, socioemotional development and family involvement. UBC’s impact 
analysis found positive impact on classroom quality with effect sizes in the moderate to large 
range (0.43-0.76) and null effects on child language and literacy skills (Yoshikawa et al, 2015). 

UBC leadership at Harvard University and Chilean Fundacion Educacional Oportunidad, 
Ministry of Education and Universidad Diego Portales proposed to employ continuous quality 
improvement methodologies (CQI) to adapt and improve the UBC intervention, in order to 
maintain the positive impacts on classroom quality and increase impact on child outcomes. 
Continuous Quality Improvement is a practical approach that helps people learn about and 
change the systems at work in their organizations. CQI requires: 1) organizational changes to 
implement improvements, and 2) frequent and continuous measurement of quality and the 
commitment to use those measures transparently to guide organizational changes. 

Organizational changes usually involve giving frontline workers, those most knowledgeable 
about the processes under study, new responsibilities and power (Kritchevsky & Simmons, 

1991), and are facilitated by formation of local CQI teams comprised of program leaders, staff, 
and recipients— in this case, school leadership, teachers, teachers’ aides and parents. 

We present the first study in South America to use CQI in the expansion of a preschool- 
based early childhood education intervention: UBC’s adoption of the Institute for Healthcare 
Improvement’s Breakthrough Series Collaborative (Institute for Healthcare Improvement, 2003). 

Purpose / Objective / Research Question / Focus of Study: 

Research Question 1 : Can the Breakthrough Series Collaborative Methodology be applied to an 
early childhood education and in Chile? 
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Research Question 2: What is the impact of the UBC professional development in instructional 
strategies with CQI methods, as compared to the UBC professional development in instructional 
strategies alone? 

Setting: 

Chile is one of South America’s most stable, prosperous and inequitable nations, with an 
average annual growth rate of 3.5 percent, the highest gross domestic product per capita in the 
region, and the largest Gini coefficient of economic inequality among OECD nations (IMF, 

2008; OECD, 2011). Inequality is observed in Chile beginning in very early stages of children’s 
development. Chilean children under age 5 from low socioeconomic backgrounds present 
significantly higher rates of socioemotional problems and language delays than children from 
families at the top of the country’s income distribution (Behnnan, Bravo, & Urzua, 2010; Schady 
et al, 2014). Chilean policymakers are concerned about the country’s large and growing 
inequality, and are well aware of the potential for early childhood education to improve 
outcomes for low-income children and to reduce societal economic inequality. Consequently, 
Chile is a global leader in early childhood policy. Since 2007, the Government of Chile 
established early childhood development policy as a key priority to close achievement gaps and 
address persistent economic inequality: it created a national integrated system for early 
childhood protection (Chile Grows with You) and expanded free early education opportunities 
for the poorest 40% of the population (Peralta, 2011; Silva & Molina, 2010). Early childhood 
education coverage for 4-year-olds expanded from 35% in 2003 to 45% in 2009 and 80% in 
2012 (Ministerio de Educacion, 2012). Despite access gains, however, concerns remain about 
preschool quality in Chile. The effectiveness of early childhood education (ECE) depends on its 
quality (Camilli et al, 2010), and several studies suggest that the quality of Chilean preschool 
environments may not be sufficient to address the disadvantages of low-income Chilean 
schoolchildren (Eyzaguirre & Le Foulon, 2001; Manzi, Strasser, San Martin, &Contreras, 2008). 

Population / Participants / Subjects: 

Forty teachers in low-income urban and rural preschool classrooms serving 1,349 four 
and five year-old children received UBC training in instructional strategies. A subset of 18 early 
adopters participated in a Learning Collaborative that taught CQI skills and facilitated shared 
learning. 

Intervention / Program / Practice: 

For all 40 participating teachers, UBC provided 12 monthly workshops and 24 biweekly 
in-classroom teacher coaching sessions to promote adoption of strategies to improve children’s 
language and literacy, with supporting interventions for children’s health, socioemotional 
development and family involvement. The 18 early adopting teachers that elected to participate 
in the Continuous Quality Improvement Learning Collaborative (CQI) also received training in 
CQI methods. Following the CQI methodology, the 18 early adopting schools fonned 
improvement teams comprised of school leadership, teachers and parents. These teams worked 
together to identify specific areas for improvement, to test ideas they had that might lead to 
improvement via Plan-Do-Study-Act (PDSA) cycles, and to collect and reflect on data regularly 
as a team to modify their practices (Langley, Moen, Nolan, Nolan, Nonnan & Provost, 2009). 

The teams also participated in three Learning Sessions, large, two-day working meetings where 
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they presented their tests of change to one another, shared learning across schools, and received 
additional training in CQI skills. 

Research Design: 

Forty schools from three municipalities in the VI Region were recruited to participate in 
an expansion phase of UBC. All schools were public, municipal preschools serving a large 
majority of “priority students,” as defined by a series of vulnerability criteria from the Ministry 
of Education. Preschool classrooms contained between 6 and 38 students, with 1 teacher and an 
additional teacher’s aide in classrooms with more than 12 students. In 201 1 and 2012, all 40 
schools received the original two-year UBC intervention, with 12 trainings and 24 in-classroom 
coaching sessions. The 18 schools that elected to participate in the CQI Learning Collaborative, 
participated in CQI skills training beginning in 201 1, and attended three Learning Sessions 
during the 2012 school year (in March, June and December). 

We assessed the feasibility and fidelity of implementation of the Breakthrough Series 
Collaborative Methodology (RQ1) using program implementation data and end-of-intervention 
interviews and focus groups. We estimated the impact of an enhanced UBC + CQI intervention 
as compared to UBC alone (RQ2) using multilevel linear regression models that accounted for 
the nesting of students within classrooms. To minimize selection bias in this non-randomized 
design, we created propensity scores from pre-intervention baseline child, family and teacher 
characteristics that predicted each child’s likelihood of participating in UBC+CQI. 

Data Collection and Analysis: 

Data for the impact analysis was collected at baseline and at the end of the second year of 
intervention. To measure classroom quality, we used a Spanish version of the Classroom 
Assessment Scoring System (CLASS; Pianta, La Paro, & Hamre, 2008), which focuses on 
interactions between students and teachers. The CLASS measure is predictive of child-level 
academic and nonacademic outcomes in prekindergarten and elementary school and is 
psychometrically validated in Chile (Leyva et al, 2015). Children’s language and early literacy 
skills were assessed using the Woodcock-Munoz Language Survey, Revised Spanish Fonn 
(WMLS-R; Woodcock, Munoz-Sandoval, Ruef, & Alvarado, 2005). 

We used pre-intervention child, family, teacher, and community characteristics to create 
propensity scores and as covariates to increase the precision of impact estimates. Parent 
questionnaires measured baseline child characteristics (gender, age, prior participation in center- 
based care, special healthcare needs, asthma) and family characteristics (maternal education and 
employment, depression, health insurance, family composition, causes of child absences, 
parents’ experiences in their child’s classroom, parents’ educational beliefs, hopes and 
expectations). Teacher surveys collected teacher demographic characteristics that are considered 
important in the literature on teacher-child interactions (Biennan, Nix, Greenberg, Blair & 
Domitrovich, 2008), including teacher age, private school teaching experience (a binary 
indicator), teaching experience (in years), and teacher postgraduate education (in years). 

We estimated the impact of an enhanced UBC + CQI intervention as compared to UBC 
alone (RQ2) using multilevel linear regression models that accounted for the nesting of students 
within classrooms. We created propensity scores from pre-intervention baseline child, family and 
teacher characteristics to predict children’s likelihood of participating in UBC+CQI, and used 
inverse propensity weights to minimize bias introduced by the non-random selection process. 
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Findings / Results: 


Research Question 1 : Can the Breakthrough Series Collaborative Methodology be applied to an 
early childhood education and in Chile? 

The Breakthrough Series Methodology for CQI was implemented well. All schools that 
participated in the Learning Collaborative fonned CQI teams that involved school leadership, 
teachers, and teachers’ aides; most included parents. All CQI teams participated in all Learning 
Sessions, conducted tests of change using PDSA cycles (see Table 1), and reported and reflected 
on quality data monthly. 

In focus groups, teachers reported that that their role, respect and visibility within the 
school had changed as a result of their work with the CQI team and because of the skills they 
developed through UBC+CQI. Many reported that prior to the program, they had felt isolated 
and sequestered in the preschool classrooms, and that because of the CQI team, school leaders 
entered preschool classrooms for the first time and were impressed that the teachers taught 
children they had thought too young to do more than play. School leaders noted that the 
preschool teachers became protagonists in the refinement of their own practices by developing 
their own hypothesis about how to improve an aspect of classroom quality and testing it through 
PDSA cycles. All schools that participated in CQI wanted to articulate the UBC instructional 
strategies and CQI methods with first grade teachers; many school principals asked the preschool 
teachers to teach their colleagues what they’d learned and some paid them for their time. School 
leaders and teachers reported that a cultural change occurred through the CQI work, away from a 
culture where quality data was used for judgment to a culture where quality data was used to 
facilitate learning and to identify opportunities for improvement. At the Learning Sessions, 
teachers shared their tests of change in which they adapted the language and literacy strategies to 
distinct contexts, and thus, they became protagonists in refining the UBC intervention as well. 

Research Question 2: What is the impact of the UBC professional development in instructional 
strategies with CQI methods, as compared to the UBC professional development in instructional 
strategies alone? 

The incorporation of CQI led to larger increases in classroom quality scores over time. The 
propensity scores that used baseline child, family and teacher characteristics to predict children’s 
likelihood of attending a school that received UBC+CQI were balanced, and they did a good job of 
reducing observable selection bias (see Table 2). Impact analyses that incorporated these 
propensity scores showed that children in classrooms whose teachers received professional 
development in instructional strategies plus CQI methods had larger increases in language skills 
(vocabulary, effect size 0.31, p<0.05) than children whose teachers received instructional 
strategies alone (see Table 3). 

Conclusions: 

CQI may be a valuable addition to promote change via professional development in low- 
income schools in Chile and warrant further examination for early childhood interventions in 
diverse contexts globally. Findings suggest that CQI works in part through changing the role of 
teachers in the school, supporting their own leadership and collaboration, and moving away from 
a culture of strict accountability (i.e. judgment) to one of facilitated and shared growth. 
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Appendix B. Tables and Figures 

Table 1. Sample Run Chart for one CQI Team’s Plan-Do-Study- Act Test of Change to Increase 
the Number of Children Using New Words Without Adult Help By Introducing One New 
Vocabulary Word Each Day 


Number of children who use the new vocabulary word 



SIN 

AYUDA 

DEL 

ADULTO 


• CON 
AYUDA 
DEL 

ADULTO 
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Table 2. Differences in baseline characteristics of children in schools that Receieved UBC + 
CQI as compared to schools that received UBC training in instructional strategies alone, before 
and after propensity score matching 


UBC+CQI UBC only UBC+CQI UBC only Si 

Baseline Variables Schools Schools Sig Schools Schools g 



39.8 


48.2 



44.9 


47.4 

(1.6 

Male 

% 

(4.3) 

% 

(1.4) 

~ 

% 

(7.8) 

% 

1) 





(0.66 


3.31 


5.4 

(0.7 

Child asthma 

1.7% 

(1.2) 

5.6% 

) 

** 

% 

(2.5) 

% 

4) 



(0.3 


(0.11 



(0.67 


(0.1 

Child age (months) 

51.6 

7) 

52.6 

) 

* 

52.8 

) 

52.4 

2) 

Maternal Education: 

13.3 



(0.84 


9.11 

(4.15 

10.0 

(0.9 

incomplete primary 

% 

(3.0) 

9.6% 

) 


% 

) 

% 

6) 

Maternal Education: 

18.8 


13.0 

(0.96 


19.7 

(6.04 

14.2 

(1.1 

complete primary 

% 

(3.5) 

% 

) 


% 

) 

% 

6) 

Maternal Education: inc. 

18.8 


18.7 



21.0 

(6.00 

19.7 

(1.2 

secondary 

% 

(3.5) 

% 

(1.1) 


% 

) 

% 

8) 

Maternal Education: 

27.3 


33.7 



23.9 

(5.81 

33.5 

(1.5 

complete secondary 

% 

(4.0) 

% 

(1.3) 


% 

) 

% 

1) 

Maternal Education: some 

21.9 


24.7 



26.2 

(7.98 

22.4 

(1.3 

superior 

% 

(3.7) 

% 

(1.2) 


% 

) 

% 

3) 


41.6 


50.4 



50.8 

(8.18 
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Maternal employment 

% 

(4.8) 

% 

(1.5) 

~ 

% 

) 

% 

2) 

Health Insurance: Public 

54.1 


64.4 



72.5 

(7.44 

64.5 

(1.5 

Tier 1-2 

% 

(4.8) 

% 

(1.4) 

* 

% 

) 

% 

5) 

Health Insurance: Public 

14.4 


25.9 



19.6 

(7.55 

24.6 

(1.3 

Tier 3-4 

% 

(3.4) 

% 

(1.3) 

** 

% 

) 

% 

6) 


31.5 



(0.75 

** 

7.85 

(2.19 

8.49 

(0.9 

Health Insurance: Blocked 

% 

(4.4) 

7.1% 

) 

* 

% 

) 

% 

9) 





(0.46 

** 

0.00 

(0.00 

2.43 

(0.4 

Health Insurance: Private 

0.0% 
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) 

* 

% 

) 

% 
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(0.03 

** 
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N Children in Household 
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1.32 
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Parents' educational 

53.9 


56.4 



51.3 


56.0 
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expectations: University 

% 

(4.4) 

% 

(1.4) 


% 

(7.7) 

% 

0) 

Parents' educational 

93.9 
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(1.4 

aspirations: university 

% 

(2.2) 

% 

(1.3) 


% 

) 

% 

0) 

Parents feel welcome in 

90.4 


79.2 


** 

90.2 

(4.43 

81.0 

(1.3 

classroom: very 

% 

(2.8) 

% 

(1.2) 

* 

% 

) 

% 

0) 

Parent has been in 

88.7 


82.7 



90.0 

(4.85 

82.2 

(1.2 

classroom 

% 

(2.9) 

% 

(1.1) 

~ 

% 

) 

% 

7) 

Teacher experience: 5 years 
or less 

Teacher experience: 6-15 

6.3% 

24.2 

(2.2) 

(3.8) 

9.6% 

25.4 

(0.84 

) 

(1.3) 


12.8 

% 

26.3 

(7.53 

) 

(6.55 

7.54 

% 

20.2 
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5) 
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years 

% 


% 



% 

) 

% 
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Teacher with postgraduate 

77.2 


37.8 


** 

58.3 

(8.25 

40.0 

(1.5 

education 

% 

(4.1) 

% 

(1.5) 

* 

% 

) 

% 

9) * 



(0.5 


(0.21 



(0.92 


(0.2 

Teacher Age 

45.4 

0) 

46.1 

) 
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) 
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2) 



(0.4 


(0.15 
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Child baseline vocabulary 
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3) 

18.3 

) 


18.1 

) 

18.0 
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Child baseline letter-word 


(0.2 


(0.07 



(0.24 


(0.0 

ID 

4.8 

3) 

5.42 

) 

~ 

4.92 

) 

5.27 
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(0.2 


(0.06 
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(0.0 

Child baseline dictation 

5.37 

0) 
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) 

* 

5.62 

) 

5.77 

7) 

Child baseline 


(0.1 


(0.03 

** 


(0.23 


(0.0 

comprehension 

2.07 

2) 

2.92 

8) 

* 

2.38 

) 

2.82 

5) 


*** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1. 
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Table 3. Adjusted Mean Language Outcomes for Children in Schools that Received UBC + CQI 
as compared to Children in Schools that Received UBC alone. 



UBC+CQI 

Schools 

SE 

UBC only 
Schools 

SE 

Difference 

Effect 

Size 

Sig. 

Vocabulary 

28.068 

0.539 

26.620 

0.158 

1.449 

0.307 

* 

Letter Word ID 

13.379 

1.132 

13.063 

0.226 

0.316 

0.129 


Dictation 

10.061 

0.345 

9.914 

0.072 

0.147 

0.073 


Passage 

Comprehension 

3.903 

0.065 

4.328 

0.354 

0.426 

0.332 
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